Stamp a node's spend on its error event, so an overspent run can say so - #84
Merged
Conversation
Tokens were attributed from `end` events only. A node the budget interrupts emits `error` instead, so the spend that triggered enforcement vanished from the run's own account of itself: `grapharc metrics` said `tokens: 0` for a run stopped with `max_tokens reached (51/5)`. Every `error` event now carries what its node spent, exactly as `end` does, and both `summarize` and the cost report count it. Sub-events inside a node stay a breakdown of its total rather than an addition, so the disjointness that kept `ends + orphans` from double-counting holds unchanged, and the `RunCost.tokens == RunMetrics.tokens` invariant the suite asserts still does. The basics cookbook stated what each phase carries and had to be re-recorded, which is the page doing its job. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #60.
The defect
Tokens were attributed from
endevents. A node the budget interrupts emitserrorinstead, anderrorcarried no token count — so the audit trail lost the spend in exactly the case an auditor cares about most.Two runs of the same one-node graph and the same scripted model, differing only in the budget:
The run was killed for spending 51 tokens and reported spending none.
The fix
Every
errorevent is stamped withctx.meter.tokens - tokens_before— what that node spent — exactly asendis. Four sites: the write/type/routing refusal and the token-ceiling refusal in_leave, and the body-raised handler in each of the sync and async wrappers. The pre-startbudget refusal is left alone: no node ran, so there is nothing to attribute.metrics.summarizeaddserrorsto its measured set, andReplayedRun.tokenscounts failed executions rather than onlyokones.On double-counting, since that is the thing to get wrong here:
NodeExecution.sub_eventsare documented as a breakdown of the node total, not an addition to it, andorphan_sub_eventsare by construction the ones no execution claimed. That is what makesends + orphanssafe today, and it applies toerroridentically. The suite'sRunCost.tokens == RunMetrics.tokensassertion still passes, and the new test asserts it directly.Tests
test_a_run_stopped_for_overspending_reports_what_it_spent— parses the figure out of the enforcement message and asserts the audit trail agrees, plus the cost/metrics equality. Confirmed red with the three source files stashed.docs/cookbook/01-basics.mddocuments what each phase carries; its transcript and prose were re-recorded, andtests/test_cookbook_basics.pyupdated to match. The page catching this is the page doing its job.Verification
pytest→ 1845 passed, 12 deselected.ruff check grapharc tests→ clean.🤖 Generated with Claude Code